Powerless signal generator for use in conjunction with a powerless position determination device

ABSTRACT

A reflector and a powerless signal generator mechanism are incorporated into a single powerless pointing device. A light source generates a light signal, and the reflector incorporated within the powerless pointing device reflects the light signal back to a image collection system of a computer. Image data is generated, and is used to generate position data relating to the position of the reflector. The position is correlated to a position on a graphical user interface. A user action activates the powerless signal generator mechanism, generating a signal. The signal can be, for example, in the form of a sound pulse having one or more predetermined frequencies or an RF signal generated by a piezo-electric transducer.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present invention is a Continuation-in-Part of U.S. patentapplication Ser. No. 11/111,444 filed on Apr. 21, 2005, and entitled“Position Determination Utilizing a Cordless Device,” which is assignedto the assignee of the current invention. The present inventionincorporates by reference U.S. patent application Ser. No. 10/739,831filed on Dec. 18, 2003 and U.S. patent application Ser. No. 10/843,517filed on May 10, 2004, both of which were entitled “Method and Systemfor Wavelength-Dependent Imaging and Detection Using a Hybrid Filter”and assigned to the assignee of the current invention.

BACKGROUND OF THE INVENTION

A conventional computer mouse functions both to track a position acrossa graphical user interface (GUI) and to input commands into thecomputer, usually relating to a word, icon, or text section on which thetracking system was located at the time of the input command. Aconventional computer mouse requires power that typically comes from oneof two sources. In a first “wired” form, a power cord tethers the mouseto the computer, thereby providing power for both the tracking functionand the input command function, while also providing a signal path forboth the position tracking system and the input command. The power cord,however, restricts freedom of movement for the user. In a second“wireless” form, an on-board power source such as a battery providespower for these functions. The signals for both the tracking and inputcommand functions are transmitted through free space in the form of RFsignals. An on-board power source, however, requires constant rechargingor replacement.

A technique for powerless position tracking is disclosed in U.S. patentapplication Ser. No. 11/111,444 entitled “Position DeterminationUtilizing a Cordless Device” filed on Apr. 21, 2005. The pointing devicedisclosed therein has a reflector and interfaces with a computer systemhaving a light source, an image collection system, a processor, and agraphical user interface (GUI). The light source generates a lightsignal and the reflector within the pointing device reflects a portionof the light signal back to the image collection system, whichcorrelates a position of the reflector to a position on the GUI. Thecorrelation is continuously updated by a processor. Because a reflectoris a passive element that does not require any power, a pointing devicethat utilizes a reflector does not need to be tethered to a computersystem or equipped with a battery. Because the “point and click”functions are so tightly interrelated when used in conjunction with aGUI, it is desirable for the “pointing” and “clicking” functions to beintegrated into a single powerless device.

SUMMARY OF THE INVENTION

A pointing device has a reflector and a powerless signal generatormechanism for generating a signal in response to a user action. Thepointing device is used in conjunction with a computer system. Thecomputer system includes an image collection system for generating imagedata which includes an indication of the reflector, a processorconfigured to use the image data to generate position informationrelated to the reflector, and a receiver configured to detect the signalgenerated by the powerless signal generator mechanism. The powerlesssignal generator mechanism includes a signal generator activated by auser action. The signal is generated to trigger a function in thecomputing system. In contrast to prior art devices, wherein the energynecessary to generate the desired signal is supplied by an externalpower source such as a battery or commercial AC power, a unique featureof the present invention is that a user action supplies the energynecessary to generate the desired signal. No additional power source isneeded to generate the desired signal. The signal generated by thepowerless signal generator mechanism is received by the receiver of thecomputer where it is converted into a digital form that can be used bythe processor to, for example, navigate within a GUI.

The powerless signal generator mechanism can be in the form of a pulsegenerator that generates a sound pulse having one or more predeterminedfrequencies, or a piezo-electric transmitter that generates an RFsignal. A pointing device may include multiple powerless signalgenerator mechanisms that are configured to generate different signals.A multi-signal embodiment allows the pointing device of the presentinvention to duplicate the functionality of a conventional mouse having,for example, “right click” and “left click” input command functions. Bygenerating command signals without the aid of an external power cord oron-board power source, the manually activated input command function canoperate synergistically with a powerless pointing device to duplicatethe functions of a conventional computer mouse, but without the need tobe electrically coupled to an external power source, nor the need for anon-board power supply such as a battery.

Other aspects and advantages of the present invention will becomeapparent from the following detailed description, taken in conjunctionwith the accompanying drawings, illustrated by way of example of theprinciples of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts a system for generating position information thatincludes a retroreflector, an image collection system, and a processor.

FIGS. 2 and 3 illustrate the detection angle, the on-axis illuminationangle, and the off-axis illumination angle in the image collectionsystem of FIG. 1.

FIG. 4 depicts light of wavelength λ1 and wavelength λ2 that is incidenton an imager after reflecting off of a retroreflector and anymiscellaneous objects.

FIG. 5 illustrates the basic operation of the system of FIG. 1.

FIG. 6 is an exemplary graph of difference image data with the intensitydifference values on the x-axis and the number of data points collectedat each intensity difference value on the y-axis.

FIG. 7 depicts an alternative embodiment of the image collection systemfrom FIG. 1 that includes one imager and multiple on-axis and off-axislight sources.

FIG. 8 depicts an embodiment of an image collection system that includestwo imagers and a single light source.

FIG. 9 depicts an embodiment of the image collection system thatincludes two imagers and light sources that provide light of twodifferent wavelengths.

FIG. 10 is a perspective view of the image collection system of FIG. 9configured in an epipolar geometry.

FIG. 11 depicts an embodiment of an image collection system thatincludes on-axis and off-axis light sources and a wavelength-selectivefilter formed over the retroreflector.

FIG. 12 depicts an exemplary embodiment of an image collection system inwhich the light sources of two different wavelengths have the same, orroughly the same, illumination angle and in which a wavelength-selectivefilter is formed over the retroreflector.

FIG. 13 depicts an embodiment of an image collection system thatincludes one imager and a single light source.

FIG. 14 depicts another embodiment of a single light source imagecollection system.

FIG. 15 depicts an embodiment of a system for generating positioninformation that utilizes a reflector other than a retroreflector.

FIGS. 16A and 16B depict an embodiment of a pointing device thatincorporates a mechanism for shielding and revealing a retroreflector.

FIGS. 17A and 17B depict another embodiment of a pointing device thatincorporates a mechanism for shielding and revealing multipleretroreflectors.

FIGS. 18A and 18B depict an embodiment of a pointing device thatincludes a retroreflector embedded into a cavity of a pointing device.

FIG. 19A depicts a top view of an imager that includes a checkerboardfilter pattern.

FIG. 19B depicts a side view of the imager from FIG. 19A with a hybridfilter over the imager.

FIG. 20 depicts a portable computer configured with an image collectionsystem that includes two light sources and one imager embedded into thehousing of the computer.

FIG. 21 depicts a portable computer configured with an image collectionsystem that includes two light sources and two imagers embedded into thehousing of the computer.

FIG. 22A a front view of an image collection system in which an imagerand light sources are incorporated into a free standing housing.

FIG. 22B depicts a side view of the image collection system of FIG. 22A.

FIG. 23 depicts a process flow diagram of a method for generatingposition information.

FIG. 24 depicts a process flow diagram of another method for generatingposition information.

FIGS. 25A and 25B depict an example of a retroreflector that has anorientation-specific shape.

FIGS. 26A and 26B depict an example of multiple retroreflectorsconfigured in an orientation-specific pattern on the surface of apointing device.

FIGS. 27A and 27B depict an example of multiple retroreflectors thathave different shapes.

FIGS. 28A-28C depict an example of a retroreflector that is integratedinto a device that has an orientation-specific structural feature.

FIGS. 29A-29C depict an embodiment of a pointing device that is similarto the pointing device described with reference to FIGS. 28A-28C exceptthat it includes a retroreflector that is not subject to theorientation-specific structural feature of the pointing device.

FIGS. 30A and 30B depict an embodiment of a pointing device in whichmultiple retroreflectors are used in conjunction withorientation-specific structural features to enable the generation ofmore detailed orientation information.

FIG. 31A depicts a side view of the pointing device from FIGS. 30A and30B in which the cavities and retroreflectors are oriented relative to abeam of light such that all of the retroreflectors are illuminated.

FIG. 31B is a front view of the pointing device from FIG. 31A in whichthe illumination of all of the retroreflectors is represented by thefilled in dark circles.

FIG. 32A depicts a side view of the pointing device from FIGS. 30A and30B in which the cavities and retroreflectors are oriented relative tothe beam of light such that only some of the retroreflectors areilluminated.

FIG. 32B is a front view of the pointing device from FIG. 32A in whichthe illuminated retroreflectors are represented by filled in darkcircles and the non-illuminated retroreflectors are represented by theun-filled in circles.

FIGS. 33A and 33B depict an example of a pointing device that utilizesan orientation-specific pattern of retroreflectors as described withreference to FIGS. 26A and 26B in conjunction with a pattern ofretroreflectors that is integrated into orientation-specific features ofthe pointing device as described with reference to FIGS. 30A-32B.

FIG. 34 depicts an embodiment of a cavity in a pointing device and aretroreflector that is attached at the bottom of the cavity.

FIG. 35 depicts a process flow diagram of a method for generatingorientation information.

FIG. 36 depicts a process flow diagram of a method for generatingposition information and orientation information.

FIG. 37 depicts a pointing device with a reflector and a powerlesssignal generator mechanism.

FIG. 38 depicts the pointing device of FIG. 37 and a computer systemthat includes an image collection system.

FIG. 39 depicts an embodiment of the pointing device of FIG. 37 whereinthe powerless signal generator mechanism is a pulse generator.

FIG. 40A depicts a side elevation view of an embodiment of the pulsegenerator of FIG. 39 prior to actuation.

FIG. 40B depicts the pulse generator of FIG. 40A during actuation, andimmediately prior to hammer release.

FIG. 41 illustrates an embodiment of the pulse generator of FIG. 39having a plurality of sonic resonators that correspond to apredetermined digital value.

FIG. 42 is a side elevation view of a piezo-electric signal generatorembodiment of the pointing device of FIG. 37.

FIG. 43 depicts an embodiment of the piezo-electric signal generatorfrom FIG. 42.

FIG. 44 depicts a process flow diagram of a method of generatingposition information relating to a pointing device.

Throughout the description similar reference numbers may be used toidentify similar elements.

DETAILED DESCRIPTION

Within the following specification, FIGS. 1-36 and the relateddescription are directed to a reflector based pointing device asdisclosed in U.S. patent application Ser. No. 11/111,444 and entitled“Position Determination Utilizing a Cordless Device,” filed on Apr. 21,2005. The focus of the present invention as found in the appended claimsis directed to FIGS. 37-44, and the description related thereto, whichincorporates a powerless signal generator mechanism described inconjunction with FIGS. 37-44 with a reflector based pointing device.

A pointing device has a reflector and a powerless signal generatormechanism for generating a signal in response to a user action. Thepointing device is used in conjunction with a computer system. Thecomputer system includes an image collection system for generating imagedata that includes an indication of the reflector, a processorconfigured to use the image data to generate position informationrelated to the reflector, and a receiver configured to detect the signalgenerated by the powerless signal generator mechanism.

The powerless signal generator mechanism includes a signal generatoractivated by a user action. The signal generated by the powerless signalgenerator mechanism is received by the receiver of the computer systemwhere it is converted into a digital form that can be used by theprocessor to navigate, for example, within a GUI.

The powerless signal generator mechanism can be in the form of a pulsegenerator that generates a sound pulse having one or more predeterminedfrequencies or a piezo-electric transmitter that generates an RF signal.A pointing device may include multiple powerless signal generatormechanisms that are configured to generate different signals in responseto different user actions. A multi-signal embodiment allows the pointingdevice of the present invention to duplicate the functionality of aconventional mouse having, for example, “right click” and “left click”input commands functions. By generating command signals without the aidof an external power cord or on-board power source, a manually activatedinput command function can operate synergistically with a powerlesspointing device, allowing the powerless pointing device to duplicate allof the functions of a conventional computer mouse, but without the needto be electrically coupled to an external power source, nor the need foran on-board power supply such as a battery.

FIG. 1 depicts a system 100 for generating position information thatincludes a retroreflector 102, an image collection system 104A, and aprocessor 106. In the embodiment of FIG. 1, the image collection systemincludes an imager 108 and first and second light sources 110, 112 thatoutput light 114, 116 having different wavelengths. The first lightsource 110 is located closer to the imager than the second light source112 and the two light sources and the imager are located in the sameplane. The first light source is referred to herein as the “on-axis”light source and the second light source is referred to herein as the“off-axis” light source.

The terms “on-axis” and “off-axis” as used herein are described withreference to FIGS. 2 and 3. The angle at which light from a light sourceilluminates the retroreflector 102 is referred to as the illuminationangle. Referring to FIG. 2, the illumination angle is measured relativeto a plane (as indicated by dashed line 120) that is coplanar with theplane of the major surface of the imager 108. The illumination angle oflight source 110 is identified as θ_(i1) and the illumination angle oflight source 112 is identified as θ_(i2). The angle at which light 118reflected from the retroreflector is incident on the imager is referredto as the detection angle. The detection angle is identified as θ_(d)and is measured relative to the same plane as the illumination angle. Inthe case of two illumination angles, θ_(i1) and θ_(i2), the term“on-axis” applies to the light source or the illuminating beam whoseillumination angle has the smaller difference from the detection angle,θ_(d), and the term “off-axis” refers to the light source or theillumination beam whose illumination angle has the largest differencefrom the detection angle. Referring to FIG. 2, the relationship of theillumination and detection angles is mathematically expressed as:|θ_(i1)−θ_(d)|<|θ_(i2)−θ_(d)|.

FIG. 3 illustrates changes in the illumination and detection angles thatoccur as the retroreflector 102 moves relative to the imager 108 and thetwo light sources 110, 112. Although the illumination and detectionangles change as the retroreflector moves, the difference between theon-axis illumination angle and the detection angle remains smaller thanthe difference between the off-axis illumination angle and the detectionangle, |θ_(i1)−θ_(d)|<|θ_(i2)−θ_(d)|. Given this relationship betweenthe illumination and detection angles, the status of a light source asan on-axis light source or an off-axis light source does not change withmovement of the retroreflector. It should be noted that thisrelationship is still maintained when the retroreflector is moved in adirection having a component perpendicular to the plane of the figure.

Referring back to FIG. 1, the image collection system 104A is used tocollect at least two sets of image data. The first set of image data iscollected in response to light at wavelength λ1 generated by lightsource 110 and the second set of image data is collected in response tolight of wavelength λ2 generated by light source 112. The first set ofimage data, referred to herein as the on-axis image data, isrepresentative of light of wavelength λ1 that reflects off theretroreflector 102 and any other objects that are illuminated by lightsource 110. The second set of image data, referred to herein as theoff-axis image data, is representative of light of wavelength λ2 thatreflects off the retroreflector and any other objects that areilluminated by light source 112. Because the on-axis illumination angleis closer to the detection angle than the off-axis illumination angle,the on-axis image data will include a stronger indication of theretroreflector relative to the off-axis image data. In this case, theintensity of light reflected by the retroreflector to the imager 108will be much greater at wavelength λ1 than at wavelength λ2. Therefore,the intensity values at data points related to the retroreflector willbe higher in the on-axis image data than at corresponding data points inthe off-axis image data. FIG. 4 depicts light 122, 124, 126 ofwavelength λ1 and wavelength λ2 that is incident on the imager afterreflecting off the retroreflector and other objects 128. While theintensity of light 126 reflected by the retroreflector to the imagerwill be much greater at wavelength λ1 than at wavelength λ2, theintensity of light 122, 124 reflected by the other objects will beroughly the same assuming that the intensity of the light emitted by thetwo light sources is roughly equal.

The difference in the intensities of light that is reflected from theretroreflector 102 at the two wavelengths is used to generate positioninformation. The basic operation of the system of FIG. 1 is describedwith reference to FIG. 5. In FIG. 5, a retroreflector 102 is connectedto a pointing device 130 that is held in a hand 132. The hand, pointingdevice, and retroreflector are illuminated by light sources 110 and 112from FIG. 1. On-axis and off-axis image data 136, 138 are collected bythe imager while the objects are illuminated by the respective lightsources. As illustrated in FIG. 5, the on-axis image data 136 includeslight that reflects off the hand, the pointing device, and theretroreflector. Because the illumination angle at wavelength λ1 and thedetection angle are similar, the on-axis image data includes a strongindication of light from the retroreflector. The strong indication oflight is represented in FIG. 5 by a dark spot. The off-axis image data138 also includes light that reflects off the hand, the pointing device,and the retroreflector. However, because the off-axis image data isobtained in response to off-axis light, the intensity of the off-axislight that reflects off the retroreflector towards the imager 108 issmall compared to the intensity of the on-axis light that reflects offthe retroreflector towards the imager. Therefore, in the off-axis imagedata, the retroreflector does not appear significantly brighter than anyof the other objects.

In comparing the on-axis image data 136 to the off-axis image data 138,the only significant difference between the two data sets (assuming thedata sets are captured simultaneously or nearly simultaneously) is theindication of the retroreflector 102. In view of this, the position ofthe retroreflector can be definitively identified by taking thedifference between the two sets of image data to produce a differenceimage 140. Because most of the data points in the two sets of image dataare the same, most of the corresponding data points will cancel eachother out, with the one exception being the data points that correspondto the retroreflector. Therefore, the difference between the on-axisimage data and off-axis image data gives a definitive indication of theretroreflector's position. As depicted in FIG. 5, the difference imagedefinitively identifies the retroreflector while excluding any of theother objects. Referring back to FIG. 1, the difference operation iscarried out by the processor 106.

Because a difference function is relied upon to generate positioninformation, it is important to generate two sets of image data that canbe efficiently compared. In one embodiment, light detected by the imageris filtered to limit the detected light to wavelength bands around thewavelengths used for illumination. For example, when light ofwavelengths λ1 and λ2 is used for illumination, a bulk filter withtransmission peaks at wavelengths λ1 and λ2 can be located in front ofthe imager to filter out light of other wavelengths (e.g., ambientlight).

The imager can be configured to distinguish light of the two wavelengthsused for illumination. This can be done, for example, by locating acheckerboard filter pattern in front of the imager. An example of ahybrid filter for an imager that includes a bulk filter with two peaksand a checkerboard filter pattern is described in U.S. patentapplications both entitled “Method and system for wavelength-dependentimaging and detection using a hybrid filter,” application Ser. No.10/739,831, filed on Dec. 18, 2003 and application Ser. No. 10/843,517,filed on May 10, 2004, both of which are assigned to the assignee of thecurrent invention and incorporated by reference herein. FIG. 19A depictsa top view of imager 108 that includes a checkerboard filter pattern. Inthe embodiment of FIG. 19A, some imager pixels are covered by filters111 (represented by the light squares) that pass light at wavelength λ1and the other imager pixels are covered by filters 113 (represented bythe dark squares) that pass light at wavelength λ2. FIG. 19B depicts aside view of the imager from FIG. 19A with a hybrid filter 115 locatedin front of the imager. The hybrid filter includes a bulk filter 117 andthe wavelength-selective filters 111, 113. In another embodiment, onlyone set of squares (e.g., either the light or dark squares) includes awavelength-selective filter. Using a hybrid filter, the imager cansimultaneously generate the two sets of image data while theretroreflector is illuminated by reflected light from both lightsources.

Alternatively, the two sets of image data can be collected sequentially.For example, the imager and light sources can be coordinated so a firstset of image data is collected while only the first light source isactivated and the second set of image data is collected while only thesecond light source is activated. The sequential activation of lightsources and collection of the image data can be controlled by theprocessor. Although one example of sequentially collecting the two setsof image data is described, other sequential collection techniques arepossible.

Although the two sets of image data essentially cancel each other out inall areas except at the retroreflector, there are usually smalldifferences in light intensity at the non-retroreflector data points.These intensity differences are insignificant in comparison to theintensity differences at the data points that correspond to theretroreflector. In an embodiment, difference processing includesutilizing a threshold function to eliminate insignificant data points.FIG. 6 is an exemplary graph of difference image data with the intensitydifference values on the x-axis and the number of data points collectedat each intensity difference value on the y-axis. Note that the datapoints can be corrected to deal with negative intensity differencevalues. As represented by the peak 144 in the graph, most of the datapoints are clustered together. These data points represent the areaswhere the two sets of image data are roughly equivalent and essentially,but not completely, cancel each other out. There is another smallercluster of data points 146 located where the intensity difference valuesare greater. This smaller cluster of data points corresponds to thelight that is reflected from the retroreflector. As described above, theintensity difference values at these data points are larger because ofthe on-axis versus off-axis effect.

In order to easily separate the insignificant intensity differencevalues (i.e., the difference values that do not correspond to theretroreflector) from the significant intensity difference values (i.e.,the difference values that do correspond to the retroreflector), athreshold difference value is established below which intensitydifference values are considered insignificant and above which intensitydifference values are considered significant. As represented by thedashed line 148 in FIG. 6, the threshold difference value is establishedat an intensity between the two clusters of data points 144, 146. In anembodiment, the threshold difference value is pre-established or setfrom characteristics of current or prior frames and is used to simplifythe difference processing. In particular, the processor can beconfigured to discard all intensity difference values that are below thethreshold difference value. Some artifacts 147 may remain after all theintensity difference values below the threshold difference value arediscarded. Discarding insignificant data points reduces the memoryrequirements of the processor and enables faster processing and lesslatency. In one embodiment, the image processing time is reduced byprocessing only portions of an image that are near to where theretroreflector was previously found.

Intensity difference values that are determined to be significant areprocessed to identify the retroreflector. For example, all of thesignificant intensity difference values are processed to identify thecenter of the retroreflector. The location of the retroreflector'scenter can then be forwarded to a computer system through, for example,a universal serial bus (USB) interface. The processor may performadditional processing including, for example, confirming that thefeatures of the retroreflector in the difference image match the knownretroreflector characteristics and confirming that a pattern ofpotential retroreflector positions in the difference image conforms to aknown pattern of positions. For example, the difference image may becompared to known image information to determine if features in thedifference image match features of the retroreflector orretroreflectors.

Characteristics of the lighting can be established to provide good imagedata. FIG. 7 depicts an alternative embodiment of the image collectionsystem 104A from FIG. 1 in which the image collection system 104Bincludes one imager 108, multiple on-axis light sources 110, andmultiple off-axis light sources 112. In this embodiment, light sources110 generate light at wavelength λ1 and are located closer to the imagerthan light sources 112, which generate light at wavelength λ2. Themultiple light sources are used to improve the field of illumination ofthe image collection system over the system of FIG. 1. The on-axis andoff-axis image data are collected and processed as described above withreference to FIG. 1. Although two examples of light sourceconfigurations are described with reference to FIGS. 1 and 7, otherconfigurations can be used.

Although one exemplary image collection system 104A is described withreference to FIG. 1, many alternative embodiments of the imagecollection system are possible. All embodiments of the image collectionsystem collect at least two sets of image data, where one set of imagedata includes a stronger indication of the retroreflector 102 than theother set of image data. As shown below, two sets of image data can becollected in many different ways.

FIG. 8 depicts an embodiment of an image collection system 104C thatincludes two imagers 108A, 108B and a single light source 110. Thesingle light source provides light at wavelength λ1 and is locatedcloser to imager 108A than to imager 108B such that the differencebetween the illumination angle and the detection angel at imager 108A issmaller than the difference between the illumination angle and thedetection angle at imager 108B. Because of the location of the lightsource relative to the two imagers, on-axis image data is collected atimager 108A and off-axis image data is collected at imager 108B. Oncethe on-axis and off-axis image data is collected, it is processed asdescribed above to generate position information.

FIG. 9 depicts an alternative embodiment of the image collection system104D that includes two imagers 108A, 108B and light sources 152, 154that provide light of two different wavelengths. In particular, twolight sources 152 that generate light at wavelength λ1 are locatednearer to imager 108A than to imager 108B and two light sources 154 thatgenerate light at wavelength λ2 are located nearer to imager 108B thanto imager 108A. With respect to light of wavelength λ1, on-axis imagedata is collected at imager 108A and off-axis image data is collected atimager 108B. With respect to light of wavelength λ2, on-axis image datais collected at imager 108B and off-axis image data is collected atimager 108A. The configuration of FIG. 9 enables two different sets ofon-axis image data and two different sets of off-axis image data to becollected. The two imagers and the two different sets of on-axis imagedata and off-axis image data can be used to generate positioninformation in three dimensions using stereo processing or simplifiedstereo processing techniques. In an embodiment, the image collectionsystem is configured in an epipolar geometry relative to theretroreflector. FIG. 10 is a perspective view of the image collectionsystem 104D of FIG. 9 configured in an epipolar geometry. Given theepipolar geometry, the x, y, and z coordinates of the retroreflector 102at position M can be calculated by first determining the positions m1and m2 of the retroreflector 102 on the imagers 108A, 108B.

In another alternative embodiment, the image collection system includesa wavelength-selective filter associated with the retroreflector. Forexample, the filter may cover the retroreflector. FIG. 11 depicts anembodiment of an image collection system 104E that includes on-axis andoff-axis light sources 110, 112 and a wavelength-selective filter 158covering the retroreflector. In the embodiment of FIG. 11, thewavelength-selective filter is transmissive at wavelength λ1 andblocking at wavelength λ2. By transmitting light at wavelength λ1 andblocking light at wavelength λ2, the wavelength-selective filterprovides a higher intensity contrast at the data points corresponding tothe retroreflector.

Because a wavelength-selective filter associated with the retroreflectorcan filter one of the wavelengths of light emitted by the light sources,the two sets of image data can be generated with the light sources atthe same illumination angle. FIG. 12 depicts an exemplary embodiment ofan image collection system 104F in which the light sources 110, 112 oftwo different wavelengths have similar illumination angles. In theexample of FIG. 12, the wavelength-specific filter 158 associated withthe retroreflector 102 is highly transmissive at wavelength λ1 andblocking at wavelength λ2. A first set of image data is collected inresponse to reflected light at wavelength λ1 and a second set of imagedata is collected in response to reflected light at wavelength λ2.Because the wavelength-selective filter associated with theretroreflector is highly transmissive at wavelength λ1 and blocking atwavelength λ2, the first set of image data includes a strongerindication of the retroreflector than the second set of image data. Thetwo sets of image data are processed as described above to generateposition information.

In another alternative embodiment, the image collection system includesone imager and one light source. FIG. 13 depicts an embodiment of animage collection system 104G that includes one imager 108 and oneon-axis light source 110. The light source generates light atwavelength, λ1, and the imager is configured with a checkerboard filter,as described above, in which half of the checkerboard squares pass lightat wavelength λ1 and half of the squares block light at wavelength λ1.Given this configuration, a first set of image data is generated inresponse to light at wavelength λ1 and a second set of image data isgenerated in response to ambient light minus light at wavelength λ1.Because the light source is on-axis, the first set of image dataincludes a stronger indication of the retroreflector than the second setof image data. In an embodiment, a wavelength-selective filter islocated between the retroreflector and the imager. Thewavelength-selective filter passes light at wavelength λ1 and blockslight at other wavelengths. As described above with reference to FIGS.11 and 12, the wavelength-selective filter provides a higher intensitycontrast at the data points corresponding to the retroreflector.

FIG. 14 depicts another embodiment of a single light source imagecollection system. The image collection system of FIG. 14 utilizes twoseparate imagers 108A and 108B to collect the two sets of image data. Inparticular, on-axis image data is collected at imager 108A and off-axisimage data is collected at imager 108B. Again, a wavelength-selectivefilter can be located between the retroreflector and the imager to passlight at wavelength λ1.

Although the reflective element is described above as a retroreflector,a reflector other than a retroreflector can be used instead of theretroreflector. FIG. 15 depicts an embodiment of a system for generatingposition information that utilizes a mirror 160 instead of aretroreflector. As illustrated in FIG. 15, the off-axis light 116 isreflected by the mirror at an angle of reflection equal to the angle ofincidence. Reflecting the off-axis light at this angle of reflectioncauses the reflected off-axis light to miss the imager 108. On the otherhand, the mirror reflects on-axis light 1 14 back towards the imager,thus enabling the generation of two sets of image data, where one set ofimage data includes a stronger indication of the reflector than theother set of image data. In this configuration, the mirror is orientedsuch that it reflects one of the two light beams back to the imager.Alternatively, the system can be configured such that the mirrorreflects light from light source 112 back towards the imager instead oflight from light source 110.

A retroreflector can be used in a pointing application, for example, asa mouse in a GUI-based computer. In an embodiment, the retroreflectorconstitutes part of a pointing device that can be held and or moved by auser. When used in a pointing application, it is desirable to have theability to emulate certain commands via the pointing device. Forexample, it is desirable to be able to convey conventional mousefunctions such as “clicking,” “dragging,” “scrolling,” and liftdetection.

Some conventional mouse functions can be emulated by configuring thepointing device and the retroreflector such that the retroreflector canbe shielded or revealed in response to an action by the user. In oneembodiment, the retroreflector is shielded or revealed by a mechanismthat is activated by the user. FIGS. 16A and 16B depict an embodiment ofa pointing device 170 that incorporates a mechanism for shielding orrevealing a retroreflector. The pointing device includes aretroreflector 102 (FIG. 16B), a cover 172, guides 174, a finger recess176, and a return spring 178. In FIG. 16A the retroreflector (not shown)is shielded by the cover. The user can reveal the retroreflector bymoving the cover with the movement of a finger engaged with the fingerrecess 176. FIG. 16B shows the cover after it has been moved to revealthe retroreflector.

FIGS. 17A and 17B depict an embodiment of a pointing device 180 thatincorporates a mechanism for shielding and revealing multipleretroreflectors. The pointing device includes a cover 182, multipleretroreflectors 102, and a linear-to-rotation conversion mechanism 184.The linear-to-rotation conversion mechanism includes an axle 186,pivoting joints 188, a slider 173, guides 174, and a finger recess 176.In FIG. 17A the retroreflectors are shielded by the cover. A user canreveal the retroreflectors using the linear-to-rotation conversionmechanism to rotate the cover. In FIG. 17B the cover has been rotatedusing the linear-to-rotation conversion mechanism to reveal theretroreflectors through holes in the cover.

In another embodiment, the retroreflector is shielded or revealed by acombination of the structure and orientation of the pointing device. Forexample, at least one retroreflector is located at the bottom of acavity defined in the pointing device such that the retroreflector isshielded from illuminating light by the cavity walls when the pointingdevice is at some orientations with reference to the illuminating lightand is exposed to the illuminating light when the pointing device is atother orientations. FIGS. 18A and 18B depict an embodiment of a pointingdevice 190 that includes a retroreflector 102 located at the bottom of acavity 192 of the pointing device. When the cavity is not aimed towardsthe illumination light, the retroreflector is shielded by the wall ofthe cavity and is not illuminated. When the cavity is aimed towards theilluminating light, the retroreflector is illuminated. FIG. 18A depictsthe pointing device oriented such that the illuminating light 194 is notincident on the retroreflector and FIG. 18B depicts the pointing deviceoriented such that the illuminating light is incident on theretroreflector.

Although some examples of techniques for shielding and revealing aretroreflector are described with reference to FIGS. 16A-18B, many otherconfigurations are possible. In another alternative embodiment, someconventional mouse functionality can be achieved by changing somecharacteristic of the retroreflector or retroreflectors to emulate amouse function. For example, the shape, size, and/or pattern of aretroreflector or retroreflectors can be changed to emulate a mousefunction. In another alternative embodiment, some functionality can beachieved by gradually revealing or shielding a retroreflector. Forexample, the scrolling speed can be increased as more of aretroreflector is revealed or decreased as more of the retroreflector isshielded.

The above-described techniques for generating position information canbe incorporated into a position tracking system for a GUI-basedcomputer. FIGS. 20-23 depict examples of an image collection system anda pointing device integrated with a computer system. FIG. 20 depicts aportable or “laptop” computer 200 configured with an image collectionsystem that includes two light sources 110, 112 and an imager 108mounted in the housing of the computer. The first light source 110 is anon-axis light source that generates light at wavelength λ1 and thesecond light source 112 is an off-axis light source that generates lightat wavelength λ2. The configuration of the image collection system issimilar to that of FIG. 1 with the processor 106 being included withinthe computer. A pointing device 130 that includes a retroreflector 102is used to indicate position in a GUI-based operating system. Positionof the pointing device is determined as described above.

FIG. 21 depicts a portable computer 200 configured with an imagecollection system that includes two light sources 152, 154 and twoimagers 108A, 108B mounted in the housing of the computer. Theconfiguration of the image collection system is similar to that of FIG.9. Given this configuration, stereo processing can be used to determinethe three-dimensional position of the pointing device.

FIGS. 22A and 22B depict front and side views of an embodiment of animage collection system in which an imager 108 and light sources 110 and112 are incorporated into a free standing housing 202. Referring to FIG.22A, light sources 110 emit light at wavelength λ1 and light sources 112emit light at wavelength λ2. Referring to FIG. 22B, the retroreflectoris covered with a wavelength-selective filter that is transmissive atwavelength λ1 and blocking at wavelength λ2. The operation of thisconfiguration is similar to the configuration described above withreference to FIG. 12. Specifically, a first set of image data iscollected in response to reflected light at wavelength λ1 and a secondset of image data is collected in response to reflected light atwavelength λ2. Because the wavelength-selective filter associated withthe retroreflector is transmissive at wavelength λ1 and blocking atwavelength λ2, the first set of image data includes a strongerindication of the retroreflector than the second set of image data. Thetwo sets of image data are processed as described above to generateposition information. The image collection system depicted in FIGS. 22Aand 22B can be used in conjunction with a computer. For example, theimage collection system can be connected to a computer through a wiredUSB connection or a wireless connection. In an embodiment, the head 203of the housing can be rotated to change the field of view of the lightsources and imager. For example, the head can be rotated downward to,for example, bring a desktop that is used for mouse navigation intoview.

FIG. 23 depicts a process flow diagram of a method for generatingposition information. At block 220, a reflector is illuminated. At block222, a first set of image data and a second set of image data aregenerated. The first set of image data includes a stronger indication ofthe reflector than the second set of image data. At block 224, positioninformation related to the reflector is generated using the first set ofimage data and the second set of image data.

FIG. 24 depicts a process flow diagram of another method for generatingposition information. At block 230, a pointing device that includes aretroreflector is illuminated. At block 232, on-axis image data isgenerated in response to the illumination. At block 234, off-axis imagedata is generated in response to the illumination. At block 236,position information related to the retroreflector is generated usingthe on-axis image data and the off-axis image data.

Because the image collection system does not rely on reflections from adesktop surface as is the case in conventional optical mouse devices,the position of the pointing device can be tracked even when thepointing device sits on a non-reflective surface or a microscopicallysmooth surface such as glass.

Although the system for generating position information is describedwith reference to pointing applications for GUI-based computer operatingsystems, the above-described technique for generating positioninformation can be used in other applications that involve determiningthe position of an object. One application includes, for example,tracking the motion of a body part for use in motion capture or gameplaying.

Another advantage of the above-described dual image position trackingtechnique is that the use of a reflector or a retroreflector allows theimage data to be collected with reduced exposure time compared toconventional digital photography exposure times.

Although the retroreflectors are described as being attached to apointing device, the retroreflectors could be integrated with some otherobjects. For example, the retroreflectors can be attached to clothing,to parts of the body, or any other object whose position and/ororientation is of interest.

An object in free space can be characterized in terms of six degrees offreedom, including three degrees of freedom in translation (e.g., the x,y, and z axes) and three degrees of freedom in rotation (e.g., rotationabout the x, y, and z axes). The position of an object can becharacterized in terms of, for example, the x, y, and z axes. Theorientation of an object can be characterized in terms of, for example,rotation around the x, y, and z axes.

The technique described above with reference to FIGS. 1-24 focusesprimarily on generating position information related to an object, forexample, the two-dimensional position of an object in terms of the x andy axes or the three-dimensional position of an object in terms of the x,y, and z axes. In some applications, it is desirable to determine theorientation of an object either alone or in conjunction with theposition of the object. In accordance with the invention, generatingorientation information involves capturing two sets of image data usingat least one reflector with an orientation-specific characteristic or areflector integrated into a device that includes someorientation-specific structural feature. The two sets of image data arethen used to generate orientation information. In particular, thedifference is taken between the two sets of image data and because theimage data is captured using a reflector with an orientation-specificcharacteristic or a reflector integrated into a device that includes anorientation-specific structural feature, the resulting difference imageincludes orientation-specific image information related to theorientation of the reflector.

Orientation-specific image information can be generated, for example,using a retroreflector with an orientation-specific shape, usingmultiple retroreflectors in an orientation-specific arrangement, or bylocating one or more retroreflectors at the bottom of respectivecavities of a pointing device such that the retroreflectors areilluminated only in certain orientations. Various examples of techniquesfor generating orientation-specific image information are describedbelow with reference to FIGS. 25A-36. In particular, FIGS. 25A-27Bdepict examples of a retroreflector or multiple retroreflectors thathave an orientation-specific characteristic, FIGS. 28A-32B depictexamples of a retroreflector or retroreflectors integrated into apointing device that includes some orientation-specific structuralfeature, and FIGS. 33A and 33B depict a pointing device that includes acombination of an orientation-specific pattern of retroreflectors and apattern of retroreflectors integrated into a pointing device thatincludes orientation-specific structural features.

FIGS. 25A and 25B depict an example of a retroreflector 300 that has anorientation-specific shape, in this case an arrow. Difference imagesformed in response to this retroreflector are unique at everyorientation of the retroreflector about the z axis. That is, theretroreflector has a rotationally asymmetric shape. In FIG. 25A thearrow points straight up and in FIG. 1B the arrow is rotatedapproximately forty-five degrees clockwise. The image of the arrow isidentified by capturing two sets of image data and taking the differencebetween the two sets of image data. The orientation of theretroreflector can be determined by comparing the difference image toknown image information. For example, the known image information is animage of the arrow in a known orientation. The known image and thedifference image can be matched to determine orientation. For example,the rotation needed to match the difference image to the known imagegives an indication of the orientation of the retroreflector. In theembodiment of FIGS. 25A and 25B, the retroreflector is located on thesurface of a pointing device 302.

FIGS. 26A and 26B depict an example of multiple retroreflectors havingthe same shape but configured in an orientation-specific arrangement onthe surface of a pointing device 302. In particular, the patternincludes three retroreflectors with a single retroreflector at the topof the pattern and two retroreflectors nearby each other at the bottomof the pattern. Difference images formed in response to this pattern ofretroreflectors are unique at every orientation of the arrangement aboutthe z axis. For example, in the orientation of FIG. 26A, the singleretroreflector is at the top of the pattern and the two nearbyretroreflectors are at the bottom of the pattern. In the orientation ofFIG. 26B, the pattern is rotated about the z axis by about ninetydegrees clockwise such that the single retroreflector is at the right ofthe pattern and the two nearby retroreflectors are at the left of thepattern.

FIGS. 27A and 27B depict an example of multiple retroreflectors 300 thathave different shapes in an orientation-specific arrangement. In theexample of FIG. 27A and 27B, the retroreflectors are in the shape of asquare, a circle, a triangle, and a diamond. Difference images formed inresponse to this pattern of retroreflectors are unique in everyorientation about the z axis because the different shapes are indifferent locations in every orientation.

FIGS. 28A-28C depict an example of a retroreflector 310 that isintegrated into a device 302 that has an orientation-specific structuralfeature. In the example of FIGS. 28A-28C, the device is a pointingdevice and the orientation-specific structural feature is a cavity 304that extends into the pointing device from one of its surfaces. Thecavity is defined by cavity walls 312 and the retroreflector 310 islocated at the bottom of the cavity. FIG. 28A depicts a side view of thepointing device that identifies the cavity and the retroreflectorlocated at the bottom of the cavity. FIG. 28B depicts a side perspectiveview of the pointing device, the cavity, and the retroreflector relativeto a beam 308 of light. In FIG. 28B, the cavity and retroreflector areoriented relative to the beam of light such that the retroreflector isilluminated by the beam of light. Light that illuminates theretroreflector is reflected back in the same direction it is received.If the orientation of the cavity and the retroreflector relative to thebeam of light changes enough, then the beam of light is blocked by someportion of the pointing device such that the retroreflector is notilluminated. In orientations other than that shown in FIG. 28B, theorientation-specific structural feature occludes the retroreflector atleast in part. FIG. 28C depicts a side perspective view of the pointingdevice in which the cavity and the retroreflector are oriented relativeto the beam of light such that the retroreflector is not illuminated.Given the configuration of FIGS. 28A-28C, a stronger indication of theretroreflector will be present in the difference image when the cavityand retroreflector are aligned with the beam of light as shown in FIG.28B than in the difference image that is generated when the cavity andretroreflector are not aligned with the beam of light as shown in FIG.28C. The rotational orientation of the retroreflector and the pointingdevice about the x and y axes can be characterized based on theexistence of the retroreflector in the difference image.

In the embodiment of FIGS. 28A-28C, the alignment of the pointing deviceis a binary characterization. That is, the pointing device is determinedto be either aligned with the beam of light when a strong indication ofthe retroreflector is present in the difference image or not alignedwith the beam of light when a strong indication of the retroreflector isnot present in the difference image. In another embodiment, theorientation between the cavity, the retroreflector, and the beam oflight can be determined from the shape of the retroreflector in thedifference image or from the intensity values of data points in thedifference image that are related to the retroreflector. For example, inone embodiment, a partial image of the retroreflector can be used todetermine orientation. In another embodiment, the intensity of thedetected light from the retroreflector can be used to determineorientation. The intensity of the detected light from the retroreflectorcan change as the distance between the retroreflector and the imagerchanges and/or as orientation-specific structural features reveal orblock the retroreflector as the orientation of the pointing devicechanges. Note that orientation can be determined from the shape of aretroreflector regardless of distance by comparing the ratio of heightto width of the image.

In the example of FIGS. 28A-28C, the orientation of the pointing device302 is determined relative to rotation around the x and y axes. Asdescribed above, the retroreflector 310 disappears from the differenceimage as the pointing device is tilted to the point where theretroreflector is no longer illuminated. In some applications it isdesirable to simultaneously track the position and the orientation of apointing device. The position of the pointing device in FIGS. 28A-28Ccannot be tracked using the above-described dual image technique whenthe retroreflector is not illuminated.

In an embodiment, an additional retroreflector that is not subject tothe orientation-specific structural feature of the pointing device isplaced on a surface of the pointing device to enable position tracking.FIGS. 29A-29C depict an embodiment of a pointing device 302 that issimilar to the pointing device described with reference to FIGS. 28A-28Cexcept that it includes a retroreflector 300 that is not subject to theorientation-specific structural feature of the pointing device.Referring to FIG. 29A, the additional retroreflector is located on theouter surface 314 of the pointing device. When the pointing device isoriented as shown in FIG. 29B, both the retroreflector 300 on the outersurface of the pointing device and the retroreflector 310 in the cavity304 of the pointing device are illuminated by the beam 308 of light.Because both retroreflectors are illuminated, they will both berepresented in the difference image. When the pointing device isoriented as shown in FIG. 29C, the retroreflector 300 on the outersurface of the pointing device is illuminated but the retroreflector 310in the cavity is not illuminated because the beam of light is blocked bya structural feature of the pointing device. Even though the orientationof the pointing device in FIG. 29C prevents the retroreflector 310within the cavity from being illuminated, the retroreflector 300 on theouter surface of the pointing device is still illuminated and can beused for position tracking. The dimensions and wall angles of the cavitycan be configured to set the field of view of the retroreflector.

FIGS. 30A and 30B depict an embodiment of a pointing device 302 in whichmultiple retroreflectors 310 are used in conjunction withorientation-specific structural features 324 to enable the generation oforientation information. In the embodiment of FIGS. 30A and 30B,multiple retroreflectors 310 are located in respective cavities. Thecavities are arranged in a circle 325. Each cavity has its ownorientation-specific sensitivity and the orientation of the pointingdevice can be determined from the locations of the retroreflectors thatare represented in the difference image. Referring to FIG. 30B, theorientation-specific sensitivity of the cavities is achieved by formingcavity walls 326 and 328 at different angles. As depicted in FIG. 30B,the walls 328 of the cavities closer to the center of the circle are ata larger angle relative to a line perpendicular to the surface 314 ofthe pointing device than the walls 326 further from the center.

FIG. 31A depicts a side view of the pointing device 302 from FIGS. 30Aand 30B in which the cavities 324 and retroreflectors 310 are orientedrelative to a beam 308 of light such that all of the retroreflectors areilluminated. FIG. 31B is a front view of the pointing device from FIG.31A in which the illumination of all of the retroreflectors isrepresented by the filled in dark circles. From the illumination patternin FIG. 31B, it can be inferred that all of the cavities andretroreflectors are aligned with the beam of light. FIG. 32A depicts aside view of the pointing device from FIGS. 30A and 30B in which thecavities and retroreflectors are oriented relative to the beam 308 oflight such that only some of the retroreflectors are illuminated. FIG.32B is a front view of the pointing device from FIG. 32A in which theilluminated retroreflectors are represented by filled in dark circlesand the non-illuminated retroreflectors are represented by the un-filledin circles. The orientation of the pointing device can be determinedfrom the identity of the illuminated retroreflectors. In thisembodiment, a non-illuminated retroreflector indicates that the pointingdevice is oriented such that the non-illuminated retroreflector hasrotated towards the image collection system. Because the retroreflectorsare located on the pointing device around the perimeter of circle 325,the orientation of the pointing device relative to the x and y axes canbe determined. In some orientations, retroreflectors may be partiallyilluminated. Additional orientation information can be obtained from,for example, the shapes and/or intensity values of the image informationrelated to the partially illuminated retroreflectors.

In another embodiment, the orientation-specific structural feature is aslot that extends into a device such as a pointing device. Aretroreflector is located within the slot. Because of the shape of theslot, the field of view of the retroreflector is larger in one directionthan in the other direction. Specifically, the field of view in thedirection parallel to the slot is larger than the field of view in thedirection perpendicular to the slot. The differences in the field ofview allow the retroreflector to be illuminated over a larger range inthe parallel direction and a smaller range in the perpendiculardirection. A change in orientation in the perpendicular direction willbe indicated by the partial or complete disappearance of theretroreflector in the difference image. Characteristics of the field ofview of a retroreflector within a slot are a function of the width anddepth of the slot. The dimensions and configuration of theorientation-specific structural feature can be set to achieve thedesired response.

Multiple techniques for generating orientation information can becombined to enable the generation of more detailed orientationinformation. FIGS. 33A and 33B depict an example of a pointing device302 that utilizes an orientation-specific pattern of retroreflectors 300as described with reference to FIGS. 26A and 26B in conjunction with apattern of retroreflectors 310 that is integrated intoorientation-specific features 324 of the pointing device as describedwith reference to FIGS. 30A-32B. The configuration of FIGS. 33A and 33Benables the generation of orientation information in terms of rotationaround the x, y, and z axes. In particular, the orientation-specificpattern of retroreflectors 300 on the surface 314 of the pointing deviceenables the generation of orientation information in terms of rotationaround the z axis and the pattern of retroreflectors 310 located in thecavities of the pointing device enables the generation of orientationinformation in terms of rotation around the x and y axes.

FIG. 34 depicts an embodiment of a cavity 324 in a pointing device 302and a retroreflector 310 located at the bottom of the cavity. The cavityhas at least two walls 326 and 328 that have different angles relativeto the axis 330 perpendicular to the surface 314 of the pointing device302. In particular, one of the walls 326 of the cavity is at an anglethat is near parallel to axis 330 and the other wall 328 of the cavityis at an angle that is non-parallel to axis 330. The angles of the wallsand the depth of the cavity define the field of view of theretroreflector. Because the angles of the walls are different, theretroreflector has different fields of view at different orientations.The different fields of view are utilized as described above to give anindication of orientation. Although FIG. 34 depicts an example of acavity, other cavity configurations are possible.

Although the cavities depicted in FIGS. 30A-33B are symmetricallyoriented about a circle, the cavities could alternatively be located ina different pattern. Additionally, the field of view of the cavities canbe in a different, for example, more random configuration.

FIG. 35 depicts a process flow diagram of a method for generatingorientation information. At block 420, a reflector is illuminated. Atblock 422, a first set of image data and a second set of image data aregenerated, wherein the first set of image data includes a strongerindication of the reflector than the second set of image data. At block424, orientation information related to the reflector is generated usingthe first set of image data and the second set of image data.

FIG. 36 depicts a process flow diagram of a method for generatingposition information and orientation information. At block 430, apointing device that includes a retroreflector is illuminated. At block432, a first set of image data is generated in response to theillumination. At block 434, a second set of image data is generated inresponse to the illumination. At block 436, position information andorientation information related to the retroreflector is generated usingthe first set of image data and the second set of image data.

Although some exemplary orientation-specific shapes and patterns ofretroreflectors are described above as examples of orientation-specificcharacteristics, other orientation-specific configurations are possible.For example, retroreflectors in a line, a pattern of retroreflectorlines, or the size of retroreflectors can be used to achieve anorientation-specific characteristic. Additionally, although someexemplary orientation-specific structural features are described above,other orientation-specific structural features are possible.

Although the retroreflectors are described as being attached to apointing device, the retroreflectors could be integrated with some otherobjects. For example, the retroreflectors can be attached to clothing,to parts of the body, or any other object whose position and/ororientation is of interest.

FIGS. 37 through 44 depict embodiments of a pointing device having apowerless signal generator mechanism configured to generate a signal inresponse to a user action. FIG. 37 depicts a pointing device 130 thatincludes both a retroreflector 102 and a powerless signal generatormechanism 502. The powerless signal generator mechanism is actuated by auser action and translates the user action to a signal. The pointingdevice of FIG. 37 can thereby duplicate the “point-and-click”functionality of a conventional computer mouse or mouse-type interfacedevice, but without being tethered to an external power supply andwithout the use of an on-board power supply such as a battery.

FIG. 38 depicts the pointing device 130 of FIG. 37 in conjunction with acomputer system 100 that includes an image collection system 104, aprocessor 106, a GUI 105, and a receiver 103. As discussed below, avariety of signal-types can be generated by the powerless signalgenerator mechanism and therefore the receiver is configured tocorrespond to the type of signal generated by the powerless signalgenerator mechanism. In operation, as a user positions the pointingdevice, the retroreflector 102 reflects light back to the imagecollection system 104 and the position of the pointing device isdetermined as described above. When the user desires to trigger aparticular function, the powerless signal generator mechanism 502 isactivated by a user action. A signal is generated by the powerlesssignal generator mechanism in response to the user action and receivedby the receiver 103. The receiver notifies the processor that aparticular function has been initiated. In this manner, the user cannavigate screen displays or initiate programs, functions and sequenceswithin the GUI similar to a conventional mouse.

FIG. 39 depicts an embodiment of the pointing device 130 of FIG. 37wherein the powerless signal generator mechanism is a pulse generator500 configured to generate a sound pulse in response to a user action.As described above in conjunction with FIG. 38, when the sound pulse isreceived by the receiver 103, the receiver initiates a signal to theprocessor 106, which triggers a function in the computer system. In anembodiment, the pulse generator utilizes a mechanically activated sonicresonator that generates a sound pulse outside the audible frequencyrange. Sonic resonators are well-known in the field and are notdescribed in detail herein. Likewise, sonic receivers that can be usedas receiver 103 (FIG. 38) are well-known in the field.

FIG. 40A depicts an embodiment of a pulse generator that utilizes amechanically actuated sonic resonator 525 to generate a sound pulse. Thepulse generator 500 includes an interface member shown as a push button501 mechanically linked to an upper half of a pivot member 505. Thepivot member is configured to pivot around a fulcrum 507 that is coupledto a fixed member 523 such as the casing of the pointing device. Thelower half of the pivot member is coupled both to a trigger spring 509,and to a trigger 513 having a trigger release 515 at its distal end. Animpulse spring 521 has a proximal end coupled to a hammer 517 and adistal end coupled to a fixed member 524 such as the casing of thepointing device. The hammer has a hammer release 519 for releasablyengaging the trigger release. Prior to activation of the sonicresonator, a reset gap 511 separates the trigger release from the hammerrelease.

The sonic resonator 525 has an anvil surface 529 oriented along a lineof travel that the hammer 517 will follow upon activation. The sonicresonator is secured in place by support members 527, which areconnected to a fixed surface 531, such as the outer casing of thepointing device. The support members are configured to optimize the rateof damping of the sonic resonator, thereby ensuring that, in operation,the sound pulse is neither damped too quickly, nor too slowly. Thesupport members are configured to minimize undesirable alterations inthe frequency at which the sonic resonator resonates.

Referring to FIG. 40B, in operation, a user engages the interface membershown as pushbutton 501. The pushbutton is depressed by pressure fromthe user's finger 503, thereby advancing the upper half of the pivotmember 505 about the fulcrum 507 and retracting the lower half of thepivot member. The lower half of the pivot member physically compressesthe trigger spring 509 against the fixed surface 523. Simultaneously,the lower half of the pivot member mechanically retracts the trigger513, forcing the trigger release 515 to engage the hammer release 519.As the actuation progresses, the retraction of the hammer progressivelydeforms the impulse spring 521, resulting in increasing flexure of theimpulse spring 521. The impulse spring is configured to flex along apath that causes the hammer release 519 to slide downward against thefront surface of the trigger release 513. The impulse spring isconnected to the fixed member 524 at a position that is configured toallow disengagement of the hammer release and the trigger release at apredetermined point of flexure of the impulse spring. FIG. 40B shows therelationship of the trigger release 515 and hammer release 519 justprior to release of the hammer.

When the depression of push button 501 reaches a predetermined distance,the trigger release 513 disengages the hammer release 519. Upon release,the impulse spring 521 impels the hammer toward the anvil surface 529 ofthe sonic resonator 525, striking the anvil surface and causing thesonic resonator to vibrate at a predetermined frequency, therebygenerating a sound pulse. The sound pulse is detected by the receiver103 shown in FIG. 38. The receiver generates an electrical signal inresponse to the detected sound pulse and provides the electrical signalto the processor 106. The electrical signal is used to trigger afunction in the computer system. For example, when a user positions apositioning image such as a cursor upon a select icon, hyperlink, ortext portion of a graphical user interface (GUI) through use of thepointing device and activates a signal from the powerless signalgenerator mechanism, a user can initiate programs, functions andsequences within the computer or viewing screen. Throughout thisdisclosure, the depiction of a push-button interface member isexemplary. The present invention comprehends alternative manualinterface devices, including, but not limited to proximity sensors,photo sensors, thermocouples, and voice activation.

Although variation exists within the range of human hearing, the upperfrequency limit of the average person is somewhere in the range of 18kHz to 20 kHz. To reduce the annoyance imposed on listeners, a sonicresonator can be utilized that is configured to resonate in thefrequency range of 20 kHz to 100 kHz, thereby falling outside theaudible range. Embodiments are envisioned, however, for the use offrequencies above 100 kHz, or below 20 kHz, including those in theaudible range.

Because many pointing devices currently in use have multiple signalgenerators, such as “right click” and “left click” mouse embodiments,the present invention can be configured with multiple sonic resonatorshaving distinct frequencies. The “left” and “right” click functions of atraditional mouse can thereby be duplicated by distinct frequencies, ordistinct frequency combinations. This can be accomplished through as fewas two sonic resonators of distinct frequencies, a first representing aleft click function and a second representing a right click function.Distinguishable pulses can be activated by distinct interface members onthe powerless signal generator mechanism or distinct actions (such as“single clicking” or “double clicking”) on a particular interfacemember.

It is possible that stray noise having a frequency matching theresonator frequency could trigger a false input signal. To reduce thelikelihood of triggering a false input signal, an embodiment furtherincorporates a receiver configured to recognize an input command signalonly if the sound volume of the incoming signal exceeds a minimum dbthreshold. If the db volume of an incoming signal does not exceed aminimum threshold volume, the receiver does not recognize an inputcommand signal. Although there are many variables, sound dissipates atroughly the inverse square of the distance from the source. By assigningan “operational distance” as the maximum distance from a computer atwhich a user is likely to operate the pulse generator, and determiningthe detected db volume of a signal generated from that maximum distance,the receiver can be configured to recognize a valid signal only if thesignal is generated within the “operational distance” of the receiver.As a result, activation of the pulse generator 500 of a pointing deviceby a first person will not interfere with a second computer outside theoperational distance.

Even using a minimum volume threshold, however, it remains possible forstray noise of a given frequency to satisfy the minimum db threshold. Anadditional safeguard against a false input command can be achievedthrough a multi-frequency pulse generator having two or more sonicresonators that are engaged simultaneously by the activation of the samepush button thereby collectively representing a single sound pulse. Thereceiver is configured to recognize the incoming pulse as a valid inputsignal only if all required frequencies are present and if thoserequired frequencies exceed a minimum db threshold. Additionally, thereceiver can be configured to recognize the incoming pulse as a validinput signal only if the db volumes of the respective frequencies arewithin a preset db range of each other. Assume, for example, a twofrequency embodiment wherein frequencies f1 and f2 are determined toreliably remain within 3 db of each other during signal propagation. Thereceiver is therefore configured to recognize a valid signal only iffrequencies f1 and f2 both exceed the minimum db threshold and only iff1 and f2 are within a 3 db range of each other. If a sound pulsecomprises f1 and f2 and the volume of both frequencies are within 3 dbof each other, the receiver will interpret the incoming signals as acommand pulse generated by the pulse generator 500. However, if therespective volumes of the two frequencies are not within the exemplary 3db range of each other, the receiver will disregard the disparatefrequencies as noise and not an input command produced by sonicresonators within the pointing device. In this way, a multi-frequencypulse can provide additional safeguards against inadvertent inputcommand activation as the result of noise.

Another advantage of using multiple sonic frequencies to signify apositive activation of a pulse generator 500 can be appreciated inforums wherein multiple persons are working with pointing devices whichare all configured with pulse generators 500. If multiple pointingdevices were to use the same frequency or frequency combination torepresent the “right click” signal of a conventional mouse, as a firstuser activated a first pulse generator 500 to trigger a response by afirst computer, the signal could have the undesirable effect ofinputting a signal to a second computer. By fabricating each pointingdevice with a plurality of sonic resonators 525, different pointingdevices can be configured to activate different combinations of sonicresonators. As long as two users who are proximate each other use adifferent combination of frequencies, no interference between the twousers will occur.

FIG. 41 depicts eight sonic resonators 550-557 that can be incorporatedwithin a single pointing device. The eight sonic resonators correspondto an eight-bit field 549. The bit field displays a predetermined set ofones and zeros for representing a positive activation in response to thefrequency pattern of a sound pulse. Although an eight bit fieldtechnically has 2⁸ possible configurations, a binary field of all zeroswould indicate that no sonic resonator had been activated. From this, itcan be readily appreciated that in a pointing device having “n” sonicresonators available at distinct frequencies, the number of possiblefrequency combinations for representing an activation signal is equal to2^(n)−1 different frequency combinations. According to an embodiment,each pointing device is user-configurable, thereby allowing responsivemeasures to be taken by a user in the event of interference withneighboring uses.

FIG. 42 depicts a pointing device 130 wherein the powerless signalgenerator mechanism is a piezo-electric signal generator 603. Thepiezo-electric signal generator is configured to generate an RF signalin response to a user action. Piezostriction is a well-knownelectromechanical phenomena whereby an electrical signal applied tocertain crystal structures causes a physical constriction ordisplacement of molecular structures within the crystal. Alternatively,by applying pressure upon certain crystals, an electric signal can begenerated. Current piezo technology allows the generation of a radiofrequency signal, including a complex signal, such as a digital 32 bitsignal, by means of a mechanically activated piezo signal generator.Radio frequency receivers that can be used as receiver 103 (FIG. 38) arewell-known in the field.

FIG. 43 depicts an embodiment of a piezo-electric signal generator 603that includes a piezo energy transducer 609 and an RF transmitter 607that are electrically coupled by signal path 619. A mechanical interface617 links an interface member, shown as push button 501, to the piezoenergy transducer. When a user depresses the push button, the force ofthe depression is transmitted through the mechanical interface 617 tothe piezo energy transducer 609, imparting a force on the piezo energytransducer which results in the generation of electrical energy. Theelectrical energy is processed by the RF transmitter into an RF signalof appropriate signal attributes. Signal attributes can include, but arenot limited to frequency, bit field size, and a pre-configured binarycode. As noted above, an exemplary signal can have a 32 bit signature.

As discussed in conjunction with FIG. 38, the piezo-electric signalgenerator is used in conjunction with a pointing device 130 having aretroreflector 102 which reflects light to the image collection system104. In operation, as the user moves the pointing device 130, theprocessor 106 correlates the position of the retroreflector 102 with aposition on the GUI 105 (FIG. 38) and generates a positioning image suchas a cursor or arrow on the GUI. When the user has positioned thepositioning image over a desired icon, image or text area on the GUI,the user engages the push button 501 to activate the piezo-electricsignal generator 603. The piezo-electric signal generator generates anRF signal that is transmitted to the receiver 103. In response, thereceiver forwards a corresponding signal to the processor to trigger afunction in the computer system.

FIG. 44 depicts a process flow diagram of a method of generatingposition information relating to a pointing device. At block 701, areflector that is incorporated into a pointing device is illuminated. Atblock 703, the image collection system generates image data thatincludes an indication of the reflector. At block 705, positioninformation relating to the reflector is generated using the image data.At block 707, a signal is generated from the pointing device usingenergy provided from a user action. At block 709, the signal isdetected. At block 711, a function is initiated in response to thedetection of the signal.

As used throughout this disclosure, the terms “computer” and “computersystem” are used in the broadest sense, and include any electronicdevice incorporating digital logic and utilizing a GUI. Examples fittingthis generic description of a “computer” include, but are not limitedto, standard PCs and McIntosh computers, digital televisions, X-boxesand game stations, PDA (pocket digital assistants), Blackberry devices,cell phones, fax and copy devices having on-board display screens, andprogrammable controllers. Although specific embodiments of the inventionhave been described and illustrated, the invention is not to be limitedto the specific forms or arrangements of parts so described andillustrated. The scope of the invention is to be defined by the claimsappended hereto and their equivalents.

1. A pointing device comprising: a reflector; and a powerless signalgenerator mechanism configured to generate a signal in response to auser action, wherein the signal is generated to trigger a function in acomputing system.
 2. The pointing device of claim 1 wherein thepowerless signal generator mechanism is a pulse generator that generatesat least one sound pulse in response to the user action.
 3. The pointingdevice of claim 2 wherein the pulse generator comprises a sonicresonator and a hammer configured to engage the sonic resonator.
 4. Thepointing device of claim 2 wherein the at least one sound pulse includesa sound pulse in the 20 kHz to 100 kHz range.
 5. The pointing device ofclaim 1 wherein the powerless signal generator mechanism is apiezo-electric signal generator that generates a radio frequency (RF)signal in response to the user action.
 6. The pointing device of claim 1further comprising a second powerless signal generator mechanismconfigured to generate a second signal in response to a user action,wherein the second signal is generated to trigger a second function inthe computing system.
 7. The pointing device of claim 6 wherein thepowerless signal generator mechanisms are configured such that the firstand second signals have a distinguishable attribute.
 8. A system forgenerating position information related to a pointing device, the systemcomprising: a pointing device comprising; a reflector; and a powerlesssignal generator mechanism configured to generate a signal in responseto a user action, wherein the signal triggers a function in a computingsystem; an image collection system configured to generate image datathat includes an indication of the reflector; a processor configured touse the image data to generate position information related to thereflector; and a receiver configured to detect the signal from thepowerless signal generator mechanism.
 9. The system of claim 8 wherein:the image collection system is configured to generate a first set ofimage data and a second set of image data, wherein the first set ofimage data includes a stronger indication of the reflector than thesecond set of image data; and the processor is configured to use thefirst set of image data and the second set of image data to generateposition information related to the reflector.
 10. The system of claim 8wherein the powerless signal generator mechanism is a pulse generatorthat generates a sound pulse in response to the user action.
 11. Thesystem of claim 8 wherein the powerless signal generator mechanism is apiezo-electric signal generator that generates a radio frequency (RF)signal in response to the user action.
 12. The system of claim 8 whereinthe pointing device further comprises a second powerless signalgenerator mechanism configured to generate a second signal in responseto a user action.
 13. The system of claim 12 wherein the powerlesssignal generator mechanisms are configured such that the first andsecond signals have a distinguishable attribute.
 14. A system forgenerating position information related to a pointing device, the systemcomprising: a computing system that utilizes a graphical user interface(GUI); a pointing device for use with the GUI, the pointing devicecomprising; a reflector; and a powerless signal generator mechanismconfigured to mechanically generate a signal in response to a useraction, wherein the signal is generated to trigger a first function ofthe GUI; an image collection system configured to generate image datathat includes an indication of the reflector; a processor configured touse the image data to generate position information related to thereflector for use in navigating within the GUI; and a receiverconfigured to detect the signal from the powerless signal generatormechanism and to provide the detected signal to the computing system totrigger the first function of the GUI.
 15. The system of claim 14wherein: the image collection system is configured to generate a firstset of image data and a second set of image data, wherein the first setof image data includes a stronger indication of the reflector than thesecond set of image data; and the processor is configured to use thefirst set of image data and the second set of image data to generateposition information related to the reflector.
 16. The system of claim15 wherein the processor is configured to take the difference betweenthe first set of image data and the second set of image data to generatethe position information.
 17. The system of claim 14 wherein thepowerless signal generator mechanism is a pulse generator that generatesa sound pulse in response to the user action.
 18. The system of claim 14wherein the powerless signal generator mechanism is a piezo-electricsignal generator that generates a radio frequency (RF) signal inresponse to the user action.
 19. The system of claim 14 wherein thepointing device further comprises a second powerless signal generatormechanism configured to generate a second signal in response to a useraction, wherein the second signal is generated to trigger a secondfunction of the GUI.
 20. The system of claim 19 wherein the powerlesssignal generator mechanisms are configured such that the first andsecond signals have a distinguishable attribute.
 21. A method forgenerating position information related to a pointing device, the methodcomprising: illuminating a reflector that is incorporated into apointing device; generating image data that includes an indication ofthe reflector; generating position information related to the reflectorusing the image data; generating a signal from the pointing device usingenergy provided from a user action; detecting the signal; and initiatinga function in response to detection of the signal.
 22. The method ofclaim 21 wherein: generating the image data comprises generating a firstset of image data and a second set of image data, wherein the first setof image data includes a stronger indication of the reflector than thesecond set of image data; and generating the position informationcomprises taking the difference between the first set of image data andthe second set of image data.
 23. The method of claim 21 whereingenerating the signal comprises impacting a resonator in response to theuser action.
 24. The method of claim 21 wherein generating the signalcomprises actuating a piezo energy transducer in response to the useraction.
 25. The method of claim 21 further comprising: generating asecond signal from the pointing device using energy provided from a useraction, wherein the first and second signals have different attributes;detecting the second signal; and initiating a second function inresponse to detection of the sound signal.